Generative deep learning in digital pathology workflows

Funding: Supported by the Sir James Mackenzie Institute for Early Diagnosis, University of St Andrews and Industrial Centre for Artificial Intelligence Research in Digital Diagnostics (grant number TS/S013121/1).Many modern histopathology laboratories are in the process of digitising their workflows. Once images of the tissue exist as digital data, it becomes feasible to research the augmentation or automation of clinical reporting and diagnosis. The application of modern computer vision techniques, based on Deep Learning, promise systems that can identify pathologies in slide images with a high degree of accuracy. Generative modelling is an approach to machine learning and deep learning that can be used to transform and generate data. It can be applied to a broad range of tasks within digital pathology including the removal of color and intensity artefacts, the adaption of images in one domain into those of another, and the generation of synthetic digital tissue samples. This review provides an introduction to the topic, considers these applications, and discusses some future directions for generative models within histopathology.PostprintPeer reviewe

Deep learning for whole slide image analysis : an overview

The widespread adoption of whole slide imaging has increased the demand for effective and efficient gigapixel image analysis. Deep learning is at the forefront of computer vision, showcasing significant improvements over previous methodologies on visual understanding. However, whole slide images have billions of pixels and suffer from high morphological heterogeneity as well as from different types of artifacts. Collectively, these impede the conventional use of deep learning. For the clinical translation of deep learning solutions to become a reality, these challenges need to be addressed. In this paper, we review work on the interdisciplinary attempt of training deep neural networks using whole slide images, and highlight the different ideas underlying these methodologies.Publisher PDFPeer reviewe

Automated analysis of lymphocytic infiltration, tumor budding, and their spatial relationship improves prognostic accuracy in colorectal cancer

Funding: Medical Research Scotland, and Indica Labs, Inc. provided in-kind resource.Both immune profiling and tumor budding significantly correlate with colorectal cancer (CRC) patient outcome, but are traditionally reported independently. This study evaluated the association and interaction between lymphocytic infiltration and tumor budding, coregistered on a single slide, in order to determine a more precise prognostic algorithm for patients with stage II CRC. Multiplexed immunofluorescence and automated image analysis were used for the quantification of CD3+CD8+ T cells, and tumor buds (TBs), across whole slide images of three independent cohorts (training cohort: n = 114, validation cohort 1: n = 56, validation cohort 2: n = 62). Machine learning algorithms were used for feature selection and prognostic risk model development. High numbers of TBs (HR = 5.899, 95% CI, 1.875 - 18.55), low CD3+ 11 T cell density (HR = 9.964, 95% CI 3.156 - 31.46), and low mean number of CD3+CD8+ T cells within 50 μm of TBs (HR = 8.907, 95% CI 2.834 - 28.0) were associated with reduced disease-specific survival. A prognostic signature, derived from integrating TBs, lymphocyte infiltration, and their spatial relationship, reported a more significant cohort stratification (HR = 18.75, 95% CI 6.46–54.43), than TBs, Immunoscore, or pT stage. This was confirmed in two independent validation cohorts (HR = 12.27, 95% CI 3.524–42.73, HR = 15.61, 95% CI 4.692-51.91). The investigation of the spatial relationship between lymphocytes and TBs within the tumor microenvironment improves accuracy of prognosis of patients with stage II CRC through an automated image analysis and machine learning workflow.PostprintPeer reviewe

Assessment of immunological features in muscle-invasive bladder cancer prognosis using ensemble learning

Funding: This research received financial support from Definiens GmbH and the Industrial Centre for AI Research in digital Diagnostics (iCAIRD) which is funded by Innovate UK on behalf of UK Research and Innovation (UKRI) [project number: 104690].The clinical staging and prognosis of muscle-invasive bladder cancer (MIBC) routinely includes the assessment of patient tissue samples by a pathologist. Recent studies corroborate the importance of image analysis in identifying and quantifying immunological markers from tissue samples that can provide further insight into patient prognosis. In this paper, we apply multiplex immunofluorescence to MIBC tissue sections to capture whole-slide images and quantify potential prognostic markers related to lymphocytes, macrophages, tumour buds, and PD-L1. We propose a machine-learning-based approach for the prediction of 5 year prognosis with different combinations of image, clinical, and spatial features. An ensemble model comprising several functionally different models successfully stratifies MIBC patients into two risk groups with high statistical significance (p value < 1×10−5). Critical to improving MIBC survival rates, our method correctly classifies 71.4% of the patients who succumb to MIBC, which is significantly more than the 28.6% of the current clinical gold standard, the TNM staging system.Publisher PDFPeer reviewe

Automated detection and classification of desmoplastic reaction at the colorectal tumour front using deep learning

Funding: This study was funded by Medical Research Scotland, the Japan Society for the Promotion of Science, the British Council and Indica Labs, Inc. who also provided in kind resource.The categorisation of desmoplastic reaction (DR) present at the colorectal cancer (CRC) invasive front into mature, intermediate or immature type has been previously shown to have high prognostic significance. However, the lack of an objective and reproducible assessment methodology for the assessment of DR has been a major hurdle to its clinical translation. In this study, a deep learning algorithm was trained to automatically classify immature DR on haematoxylin and eosin digitised slides of stage II and III CRC cases (n = 41). When assessing the classifier’s performance on a test set of patient samples (n = 40), a Dice score of 0.87 for the segmentation of myxoid stroma was reported. The classifier was then applied to the full cohort of 528 stage II and III CRC cases, which was then divided into a training (n = 396) and a test set (n = 132). Automatically classed DR was shown to have superior prognostic significance over the manually classed DR in both the training and test cohorts. The findings demonstrated that deep learning algorithms could be applied to assist pathologists in the detection and classification of DR in CRC in an objective, standardised and reproducible manner.Publisher PDFPeer reviewe

Data-analysis strategies for image-based cell profiling

Image-based cell profiling is a high-throughput strategy for the quantification of phenotypic differences among a variety of cell populations. It paves the way to studying biological systems on a large scale by using chemical and genetic perturbations. The general workflow for this technology involves image acquisition with high-throughput microscopy systems and subsequent image processing and analysis. Here, we introduce the steps required to create high-quality image-based (i.e., morphological) profiles from a collection of microscopy images. We recommend techniques that have proven useful in each stage of the data analysis process, on the basis of the experience of 20 laboratories worldwide that are refining their image-based cell-profiling methodologies in pursuit of biological discovery. The recommended techniques cover alternatives that may suit various biological goals, experimental designs, and laboratories' preferences.Peer reviewe

Screening out irrelevant cell-based models of disease

The common and persistent failures to translate promising preclinical drug candidates into clinical success highlight the limited effectiveness of disease models currently used in drug discovery. An apparent reluctance to explore and adopt alternative cell-and tissue-based model systems, coupled with a detachment from clinical practice during assay validation, contributes to ineffective translational research. To help address these issues and stimulate debate, here we propose a set of principles to facilitate the definition and development of disease-relevant assays, and we discuss new opportunities for exploiting the latest advances in cell-based assay technologies in drug discovery, including induced pluripotent stem cells, three-dimensional (3D) co-culture and organ-on-a-chip systems, complemented by advances in single-cell imaging and gene editing technologies. Funding to support precompetitive, multidisciplinary collaborations to develop novel preclinical models and cell-based screening technologies could have a key role in improving their clinical relevance, and ultimately increase clinical success rates

Corrigendum:Deep Learning for Whole Slide Image Analysis: An Overview (Front. Med. (2019), 6, (264), 10.3389/fmed.2019.00264)

Equal contribution was incorrectly attributed to the authors. Instead, the statement “These authors have contributed equally to this work” should be removed.In the published article, Peter D Caie was incorrectly listed as the corresponding author. Instead, Neofytos Dimitriou should be the corresponding author. The corresponding author’s email address should read [email protected]. The authors apologize for these errors and state that these do not change the scientific conclusions of the article in any way. The original article has been updated.</p